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ABSTRACT 


Concatenated codes have long been used as a 
practical means of achieving long block or con- 
straint lengths for combating errors on very noisy 
channels. The inner and outer encoders are nor- 
mally separated by an interleaver, so that decoded 
error bursts coming from the inner decoder are 
randomized before entering the outer decoder. In 
this paper we exanine the effectiveness of this 
interleaver by calculating the cut-off rate of the 
"outer channel 1 ' seen by the outer decoder with and 
without interleaving. The results show that 
interleaving can never hurt the performance of a 
concatenated code, and that when the inner code 
rate is near the cut-off rate of the "inner chan- 
nel", interleaving can significantly improve code 
performance. 

I. INTRODUCTION 

Forney [1] first introduced concatenated 
codes as a practical means of implementing codes 
with long block or constraint lengths. Further 
use has shown these codes to be extremely powerful 
as means of combating errors on very noisy chan- 
nels. In this paper, we examine the performance 
of concatenated coding systems as measured by 
their effective channel cut-off rate. 

Concatenated coding systems are usually im- 
plemented by employing two levels of coding, as 
illustrated in Figure 1 below. 



Figure l. Concatenated Coding System 
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Binary data from the information source is 
serially partitioned into K-bit blocks that are 
subsequently used as input signals to a 2 K -ary 
block encoder known as the outer encoder. Usually 
0-2^-ary Reed-Solomon (RS) codes are used for this 
purpose. The output of the RS encoder (Q-ary sym- 
bols) is converted back into bits and serially en- 
coded by a second encoder (the inner encoder), 
which may be either block or convolutional, and 
the resultant sequence of channel symbols is sent 
over the physical channel. Decoding is accom- 
plished in the reverse order. 

For purposes of illustration, we consider the 
inner channel to be a Binary Symmetric Channel 
(BSC) derived from forcing hard decisions on an 
additive white gaussian noise (AWGN) channel. 
This channel is representative of the deep space 
channel where concatenated codes have met with a 
great deal of success. The outer channel (the 
channel presented to the RS code) is no longer 
memoryless, but has been transformed into a non- 
uniform (time-varying) channel by the inner de- 
coder. To calculate the performance of the over- 
all concatenated coding scheme requires investi- 
gating the channel produced by the inner encoder- 
BSC-inner decoder combination which Forney [1] has 
called the "super channel". 

II. CHANNEL MODELS 

McEliece and Stark [2] have suggested some 
models for channels with "block interference". In 
a block interference channel, the noise statistics 
are assigned to be constant for the time required 
to send K bits of data, but vary independently 
from one block of K bits to the next. In parti- 
cular, McEliece and Stark have evaluated the chan- 
nel capacity and cut-off rate for a two state 
block interference channel. When the channel is 
in the quiet state, no errors are made in trans- 
mission. The noisy state is represented by a BSC 
with a crossover probability of s. (A totally 
noisy channel has s*l/2) . However, even when the 
channel is in the noisy state, it is possible to 
receive the K bits of data correctly with prob- 
ability (l-s)K. In this model, the probability of 
being in a particular state, p(s), depends on the 
physical channel (fading, burst noise, frequency 
hopping, etc) and is fixed and independent of the 
block length K. 

To evaluate the performance of a concatenated 
code, we propose a modified form of the above 
model that provides a better match to this coding 
application. We assume that a rate R*K/N block 
code is used on the inner channel and that a de- 
coding error occurs with probability Pbe( r ) • That 
is, the channel is in the noisy state with this 
probability. When the channel is in the noisy 
state, each information bit is decoded incorrectly 
with probability s. Various methods can be used 
to estimate the decoded information bit error rate 










from the block error probability (see, for exam- 
pie, Clark and Cain [3]). For non-systematic 
codes, s*l/2 gives a good estimate for the decoded 
information bit error rate, given that a block 
error has occurred. For systematic codes, an im- 
proved estimate is s*d/N, where d is the minimum 
distance of the code. The channel is in the quiet 
state (s*0) with probability 1 -Pbe( r )» and no de ~ 
coding errors are made. With probability P B e( k ) 
the channel is in the noisy state and decoding 
errors are always made. The important difference 
between this model and that used by McEliece and 
Stark is that in this model the probability of be- 
ing in the noisy state is dependent on the length 
of the interference'* blocks (information word 
length) . 

When convolutional codes are used on the 
inner channel, a more complex model of the super- 
channel is needed to compute performance measures. 
The reason is that error events in convolutional 
coding have different lengths. Forney [4] has de- 
rived some random coding results on the lengths of 
these error events. To illustrate this point, 
consider the error environment generated with an 
(N,1,K) (IN output length K shift register convo- 
lutional code). Using hard decisions on the AWGN 
channel, Viterbi [5] shows that the decoded event 
error probability at any time during maximum 1 ike- 
lihood decoding is bounded by 

P(E) < T(X) | (1) 

lx - 2 /p(l-p), 

where T(X) is one form of the code generating 
function, and p is the crossover probability 
forced by using hard decisions. The decoded in- 
formation bit error probability is bounded T>y 


For inner code rates R<Roi» the random coding 
bound on the block error probability is given by 

-N(Roi-R) -K[(Roi/R)“ll 
P BE (K> <2 "2 , (5) 

where N, K, and R are the block length, informa- 
tion block length and code rate of the inner code, 
respectively. For non-systematic codes, the de- 
coded information bit error rate for the block 
code i s e qual 1/ 2 ~ g iv en that a block decoding 

error has occurrred. Therefore, the unconditional 
bit error probability of the decoded output is 

-l-K[Roi/R)-H 

Pb - (l/2)-P BE (K) < 2 . (6) 

We now proceed to evaluate the cut-off rate 
of the outer code. Two possibilities are con- 
sidered. If full interleaving is used, then the 
outer channel can be treated as a memoryless chan- 
nel and the cut-off rate can be calculated in the 
usual way. If interleaving is not used, then the 
channel is treated as a 2^-ary memoryless channel 
and the cut-off rate is evaluated by methods to be 
presented in the next section. 

A. Interleaving 

If we interleave the bits entering the super- 
channel, the channel seen by the outer coding sys- 
tem becomes a memoryless BSC with a bit error 
probability given by eqn. (6) above. Note that we 
are still using coding, so that the block error 
probabilities given are unchanged. Under this 
assumption, we calculate the cut-off rate for the 
outer channel by 

Rqo * l-log 2 [H-2i/Pb(r-PbT] 


. , 3T(X,Y) 

Pb(E) < ! 


(2) 

> l-log 2 H 

, /-K((R0i/R)-l] -K((R<,i/R)-l] 

]/2 • (2-2 ) 

“ 3Y i 

X - 2/p(l-p), Y-l. 


L 

J 

(7) 


For small values of p (high signal- to-noise 
ratios), the most likely error event (when the 
all-zero sequence is sent) is that the minimum 
weight path is decoded instead of the all-zero 
path. If the information sequence corresponding 
to the minimum weight path has weight b and length 
L, then the typical event error causes b bit 
errors and has length L. Therefore, we can use 
the above block code model with "block length"*L, 
and s*b/L. For larger values of p (lower signal- 
to-noise ratios), longer event errors become more 
likely and cannot be ignored. In this case a more 
general channel model for the outer channel is re- 
quired to evaluate the cut-off rate. 

III. THE CUT-OFF RATE 


Some sample calculations of Ro 0 are presented in 
Table 1 below. 
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.5109 
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.9853 
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The objective is to find the maximum achiev- 
able coding rate for the concatenated coding sys- 
tem. Define the cut-off rate of the inner channel 
as R<)i and that of the outer channel as Rqq. Now 
if a rate R code is used on the inner channel, the 
maximum overall code rate is given by 

^max “ R * R0o* (3) 

If the crossover probability of the inner BSC is 
p, then the cut-off rate, Roi, is given by [6]j 

Roi - l-log 2 E 1+2/pfl-p) ] . (4) 


Table 1. Rqo with Interleaving 


The outer channel cut-off rate is calculated as a 
function of the ratio of the inner code rate to 
the inner channe 1 cut-o f f rate, and the informa- 
tion word length K. The probability of being in 
the noisy state (block decoding error) is obtained 
from the random coding bound (5) for the parti- 
cular values of K and R/Roi in the table. Table 2 
gives the maximum attainable coding rate of the 
overall system, normalized by the inner channel 
cut-off rate, Roi* 


f(2li.s”0) - 1 , jr " x 


( 10 ) 


R/ R 0i \ 

K 4 

a 

16 

32 

64 

128 

256 

512 

0.1 

.1000 

.1000 

.1000 

.1003 

.1000 

.1000 

.1003 

.1000 

0.2 

.1984 

.2000 

.2000 

.2000 

.2000 

.2000 

.2000 

.2000 

0.3 

.2766 

.2991 

.3000 

.3000 

.3000 

.3000 

.3000 

.3000 

0.4 

.3064 

.3874 

.3998 
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.4000 

.4000 

.4000 

.4000 

0.5 

.2846 
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.4960 

.5000 

.5000 

.5000 

.5000 
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0.6 

.2270 
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.5702 

.5992 
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.6000 

.6000 

.6000 

0.7 

.1530 
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.5756 

.6878 

.6999 

.7000 

.7000 

.7000 

0.8 

.0800 

.2141 

.4554 

.7023 

.7936 

.8000 

.8000 

.8000 

0.9 

.0234 

.0749 

.2066 

.4598 

.7528 

.8868 

.8999 

.9000 


■ 0 , jr * X 

If there is a block decoding error, then we assume 
that the event {£~x) never occurs. In this case, 
the entries for each element in the channel tran- 
sition matrix for which must be normalized by 
the probability that y^c. Let a ■ l-(l-s) K be the 
probability that tor a block of K bits. Then 

_ si(l-s) K ^ 

f (zJ:!!L ,s *o) ■■ — » Z ^ i 

a 

( 11 ) 

- 0 , X * * 


Table 2. Bmax/^Oi w * th Interleaving 

The maximum values for each value of K are under- 
lined. For a given value of K, note that the max- 
imum value of Rmax/^kJi does not occur at the low- 
est value of R (which presents the best channel to 
the outer coder). Instead, there is a local maxi- 
mum for each K that drifts toward the higher rates 
as K increases. This is reasonable since the code 
performance should improve for increasing block 
lengths as long as the coding rate R is below Ro^. 
In addition, Rmax increases as R increases, given 
that K is sufficiently large. For the shorter 
block lengths (e.g,, K«4 or K»8) a lower rate 
inner code is required to "clean up" the inner 
channel, and as a consequence the overall coding 
rate suffers. 

The limiting values are of interest. From 
Table 1, we see that Rqq approaches 1 as K 
increases, and from Table 2 we note that Rmax/^Oi 
tends toward R/Roi with increasing K. When K is 
small, lower inner code rates R yield a higher 
overall coding rate. 

B . Inner Channel as a 2 K -ary Memory less Channel 

If no interleaving is used, it is difficult 
to justify using a random coding bound of the Ro 
type since the channel is not memoryless. How- 
ever, if we think of the Kth extension of the 
channel as producing a memory less channel whose 
input and output symbols correspond to blocks of 
length K, then we can calculate the channel cut- 
off rate for this memoryless 2 K -ary channel. Fol- 
lowing Massey [6] we have 

R Oo * ®ax l-logMl^Z/RZEToU)] 2 ]} (8) 
Q(x) £x — — 

where x and ^ are the channel input and output K- 
tuples, respectively, f(jr|x) is the channel tran- 
sition probability, 0(x) iT the channel input dis- 
tribution, and M*2 k . ~The calculation in eq. (8) 
begins by first calculating 

f( Z |x) - ip(s)f( Z |x,s) (9) 
s 

That is, f(£|x) is calculated for an average chan- 
nel. (This assunes that the receiver has no "side 
information" about the state of the channel, as 
defined by McEliece and Stark [2]). 

As an example, consider a two state channel 
( s*0 <■> "quiet", and s*0 <■> "noisy 11 ). Let 
Pr(s*0) - PbeOO-s, and Pr(s-O) - 1-P B E<K) - 1-e. 
Given that no block error occurs (s-0), 


where i * d B (x,^) ■ Hamming distance between x and 

z - 

The average channel transition probability is 
obtained as follows: 


f(z|x) » e f(^U,s^0) + (l-e)f(£|x,s-0) 


- (1-e) , £ - x < 12) 

* — s i (l-s) K “ 1 , y * x. 

a “ 


it is easy to 
and therefore 
Substituting 






If eq. (12) is put into matrix form, 
see that the channel is symmetric, 
q(x)*l/M maximizes Roo in eq. (8). 
in eq. (8) we obtain 

1 ( v 1 f K 

* (13) 

Since the term inside of the square brackets is 
independent of jr, the summation over £ reduces to 
a multiplication by M, and 


si(l-s)K-i 


r K /K 

: x / e _ . 

; /I -+l (. 


i-1 

/ V <* 


(14) 


-N(Roi-R) 

With e-P BE (K) < 2 , and choosing s*l/2 to 

represent the Tecoding of a non-systematic code, 
we have f 


2 

ROo>l log 2 

K l 



-K[(Roi/R)-l] 2-<K/2)((R0i/R)-2] 

\J 1-2-K 


(15) 


1 


Some sample calculations of Rqq are shown in Table 
3. The corresponding values for H^x/Roi are 
found in Table 4. As in the case of interleaving, 
Rqo approaches 1 for R/Roi < 1/2, and in all cases 
increases with increasing block length for fixed 
rate. With K fixed, Rqo decreases with increasing 
H/Roi • 


R / R 0i 

\ K 4 

8 

0.1 

1.000 

1.000 

0.2 

.9892 

1.000 

0.3 

.8981 

.9912 

0.4 

.7190 

.9197 

0.5 

.5233 

.7507 

0.6 

.3522 

.5479 

0.7 

.2143 

.3648 

0.3 

.1076 

.2136 

0.9 

.0304 

.0916 


16 

32 

64 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.9999 

1.000 

1.000 

.9891 

.9996 

1.000 

.8750 

.9375 

.9687 

.6403 

.6645 

.6666 

.4212 

.4284 

.4286 

.2473 

.2500 

.2500 

.1100 

.1111 

.1111 


128 

256 

512 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.CCC 

1.000 

1.000 

1.000 

.9844 

.9922 

.9961 

.6667 

.6667 

.6667 

.4286 

.4286 

.4286 

.2500 

.2500 

.2500 

.1111 

.1111 

.1111 


Table 3. 


Rq 0 without Interleaving 
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0.1 
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0.2 

.1978 
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0.3 

.2694 

.2974 

0.4 

.2876 

.3679 

0.5 

.2616 

.3754 

0.6 

.2113 

.3287 

0.7 

.1500 

.2554 

0.8 
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.1709 

0.9 

.0273 

.0824 


16 

32 

64 
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.2000 

.2000 
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.3000 
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.3956 

.3999 
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■ 4375 
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.4844 

.3842 
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.2948 
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.2000 

.2000 

.0990 
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.1000 


128 256 

.1000 .1000 .1000 
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Table 4. R^ax/BOi without Interleaving 

From Table 4 we see that the normalized value of 
Rmax also increases with K for a fixed value of 
R/R<)i. However, for a fixed K, the value of 
Rmax/I*0i does not mono tonic ally decrease with 
R/R()i» bu t increases to a maximum value at 
R/Roi*l/2, and then decreases as the inner code 
rate increases. For a fixed K, there is some 
optimum value of R/Roi that maximizes the total 
overall coding rate achievable. 



Figure 3. R max ^ R oi without Interleaving 


It is of interest to note the asymptotic be- 
havior of Rqq as K becomes large. If we take the 
limit as K*** of R<) 0 in eq. (15), we see that 
2 ( -(K/2)[(Roi/R)-2]) 

Rqo ♦ 1- - log 2 jl+2 \ (16) 


From Eq. (16) above, it is easy to see that 



R 0o 1 


when 

r < 

2 


and 

Roo * 

Kpi _ ^ 
R 

when 

< R < Roi- 

(17) 

Therefore, 
Rmax * 

R 

when 

R < ^21 
2 


and 

^max * 

8 

H* 

1 

when 

Roi 

— < R < Roi- 

(18) 


Figure 2 shows the normalized values of Rmax when 
interleaving is used. Figure 3 shows the normal- 
ized Rm ax when interleaving is not used. 



IV. DISCUSSION OF RESULTS 

From Figures 2 and 3 we 9ee that there is 
some optimum inner code rate R which maximizes the 
overall coding rate achievable. When interleaving 
is used, this optimum inner code rate increases 
with increasing K, approaching the cut-off rate 
Roi of the inner channel as K becomes large. On 
the other hand, when interleaving is not used, the 
optimum inner code rate is always £ 1/2 Roi. In 
both cases, the maximum overall coding rate 
achievable increases with increasing K, approach- 
ing a maximum of Roi with interleaving and 1/2 Roi 
without interleaving. 

It is instructive to compare the interleaved 
and non-inter leaved cases on the same graph. 
Figure 4 shows both the interleaved case (dotted 
curves) and the non- interleaved case (solid 
curves) for three different values of K: K*8, 
K»32, and K*“. We see that the maximum achievable 
coding rate Rmax * 9 larger with interleaving than 
without interleaving for any finite K and all 
inner code rates R. The difference is parti- 
cularly significant when R > 1/2 Roi- For large 
K(K-*») , the two curves are merged when R*l/2 Roi, 
but when R > 1/2 Roi, Rmax * *Oi for the inter- 
leaved curve whereas Fmax * 0 for the non- 
interleaved curve. These results clearly indicate 
that interleaving should be used between the inner 
and outer codes in order to maximize the overall 
achievable coding rate. 
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The situation may not be as clear if we con- 
sider the channel capacity of the outer channel 
instead of the cut-off rate. McEliece and Stark 
[2] have shown that in the case of a block inter- 
ference channel with a fixed probability (inde- 
pendent of K) of being in the noisy state, inter- 
leaving causes the channel capacity to decrease 
while the cut-off rate increases. An analysis of 
some specific codes led them to the conclusion 
that capacity was a better measure of 
code performance in that case, and hence that 
using a code designed to correct burst errors 
gives better performance than a random- 
error-correcting code with interleaving. We are 
currently investigating channel capacity for the 
concatenated code case, and hope to report the re- 
sults of this research in the near future. 

Some other areas under current investigation 
include extending the above results to the case of 
convolutional inner codes and the use of side 
information obtained from the inner decoder by the 
outer decoder. As pointed out in Section II, the 
model must be modified to include more than 2 
states when the inner code is convolutional, since 
the error events are of different lengths. In the 
case of side information, the inner decoder can 
deliver a reliability indicator along with each 
decoded block to the outer decoder. We are cur- 
rently investigating the effects these alterations 
in the model will have on the calculations of cut- 
off rate and capacity, and on the performance of a 
concatenated coding system both with and without 
interleaving. 


